This invention relates to converting the frame rate of content to a higher frame rate.
A range of different frame rates are used in the distribution and display of content. Film-based content (e.g. made-for-cinema films) typically has a frame rate of 24 Hz. Video content, such as made-for-television content, typically has a frame rate of 50 Hz or 60 Hz. A frame rate of 50 Hz or 60 Hz is typically used for the distribution of content, such as broadcast systems and recorded media.
Content with a frame rate of 24 Hz can be up-converted to the 60 Hz frame rate used for distribution by a process known as 3:2 pulldown (also known as 2:3 pulldown). This is shown in FIG. 1. An original sequence of film frames A-E at a rate of 24 Hz is shown on the upper line. The lower line shows a sequence of frames output at 60 Hz. Frames A, C and E are each shown three times, while frames B and D are each shown twice. Each frame in the pulldown sequence is a copy of the original frame. The 3:2 pulldown sequence gives a rate of 2.5 times the original frame rate, i.e. 2.5×24 Hz=60 Hz. However, the repetition of frames, and uneven rate at which frames are repeated, can be noticed by a viewer as moving objects do not follow their correct temporal position. This deficiency is known as motion judder.
Many displays now use a higher frame rate of 100 Hz or 120 Hz in order to reduce motion blur. The input content can be video content at a rate of 50/60 Hz or film content, which can either be received at a rate of 24/25/30 Hz or at a rate of 50/60 Hz, with the original 24 Hz frame rate having been up-converted to 50 Hz/60 Hz. The problem of up-converting film content which originated at 24 Hz, and which needs to be displayed at a higher rate of, for example, 120 Hz will be addressed here.
FIG. 2 shows one possible way of up-converting film-based content from 24 Hz to 120 Hz. In FIG. 2 time is shown along the horizontal axis and the position of a moving object is shown along the vertical axis. Straight line 20 indicates the true path of the moving object. At time t=0 the object is at the position labelled A, in frame A. At time t=1, the object is at the position labelled B, in frame B. The original frames A, B of the film (i.e. the original frames which occurred at the 24 Hz rate) are extracted and displayed. Motion estimation and motion-compensation techniques are then applied to calculate four intermediate frames I1-I4. For example, interpolation techniques can be used to calculate the intermediate frames by using the original frames A and B. The position of the object, as calculated at the times 0.2, 0.4, 0.6, 0.8, is also shown. As the objects lie on a straight line 20 between the two original positions A, B the viewer sees a smooth moving object. While the method shown in FIG. 2 can provide a smooth output at the higher frame rate, it can require significant hardware resources and may require an unacceptably long processing time (high video latency). This can make the technique unsuitable for certain applications, especially where a rendering device has limited resources such as memory, memory bandwidth, processing budget and power (e.g. battery life).
FIG. 3 shows another possible way of up-converting the frame rate of film content to 120 Hz which requires fewer resources of a rendering device. Motion compensated processing up-converts the frame rate to 60 Hz and then each generated frame at the 60 Hz rate is repeated, to give the output frame rate of 120 Hz. In FIG. 3, original frame A is displayed at t=0 and again at t=0.2. A first intermediate frame I1, calculated with the temporal position of the object at t=0.4, is output at t=0.4 and t=0.6. A second intermediate frame I2, calculated with the temporal position of the object at time t=0.8, is output at t=0.8 and t=1.0. This allows up-conversion with less resources, and lower latency. A drawback is that the moving object is not perceived as sharply (motion blur). Another drawback is that one out of every two original frames cannot be shown. In this example, frame B is lost because the object in frame B is at the wrong temporal position. Another drawback is that high frequencies at high contrast are perceived to flicker. The sharp originals and less sharp interpolated objects flicker at the low repeat frequency (12 Hz).
The present invention seeks to overcome at least one of these disadvantages.